Exposure control apparatus and exposure control program for vehicle-mounted electronic camera

ABSTRACT

In an exposure control apparatus for exposure control of a vehicle-mounted camera which captures successive images of a scene ahead of the vehicle, two different regions (sets of picture elements) in each image are selected for use in measuring the brightness of an object such as a preceding vehicle and the brightness of the road surface, respectively. The camera exposure is controlled based upon both of these brightness measurement results.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on and incorporates herein by referenceJapanese Patent Application No. 2007-334935 filed on Dec. 26, 2007.

BACKGROUND OF THE INVENTION

1. Field of Application

The present invention relates to an exposure control apparatus, forexposure control of an electronic camera which captures successiveimages expressing an object such as a preceding vehicle, which islocated ahead of a vehicle in which the camera is installed.

2. Description of Related Art

In recent years, vehicle-installed electronic cameras (in general,digital video cameras, referred to in the following simply as cameras)have come into use for capturing images of a region located ahead of thevehicle, with technology having been developed whereby the capturedimages (that is sets of digital data expressing respective capturedimages) are processed for detection of objects such as a precedingvehicle. The processing results can be used to generate warningindications to a vehicle driver, control driving of the vehicle, etc. Avehicle having such a camera and processing apparatus installed therein,which are being described, is referred to in the following as the “localvehicle”.

With such technology, it is important that the exposure of the camera beappropriately controlled in accordance with variations in the brightnessof the scene captured by the camera, in order to maximize thereliability of recognizing an object such as a preceding vehicle whichmay appear in an image obtained from the camera.

As described for example in Japanese patent first publication No.6-253208 (designated as reference document 1 herein), a method of usingcamera images for recognition of white lines on the road surface hasbeen proposed whereby two laterally extending sections are selectedwithin each image. A first one of these sections is positioned tocontain a part of the (imaged) road surface that is currently close tothe local vehicle. The data obtained from the first section, in each ofsuccessive captured images, are utilized for recognition of white lineson the road surface. The second section is positioned to contain a partof the road surface that is farther ahead of the local vehicle (i.e., isin an upper part of each captured image). Hence the second sectioncontains a region which will be subjected to recognition processing at afuture time point, determined by the speed at which the local vehicle istravelling. Designating the average brightness levels of the first andsecond sections as b0 and b1 respectively, the difference between theseis obtained for each of successive captured images. If the difference isfound to exceed a predetermined threshold value, then the cameranexposure which will be applied in capturing the next image is adjustedbased on the brightness value b1 (i.e., by changing the camera shutterspeed, etc).

With the above method of reference document 1, if for example the roadsurface ahead of the vehicle changes between a brightly sunlit conditionand a shade condition, the cameran exposure can be appropriatelycontrolled for each of successive captured images, i.e., such as toprevent the abrupt change in scene brightness from affecting thereliability of white line detection.

However in an actual road environment, the brightness of the roadsurface will not generally change between a sunlit condition and a shadecondition with the change extending uniformly across the road surface ina simple manner. Instead, the changes can take various forms. For thatreason, it is difficult to reliably control the cameran exposure by sucha method under actual operating conditions.

It has also been proposed, for example in Japanese patent firstpublication No. 2005-148308 (designated as reference document 2 herein)to use an exposure control apparatus whereby the brightness of the roadsurface ahead of a local vehicle is measured for use in cameran exposurecontrol, while excluding the effects of white lines (traffic lanemarkers) formed on the road surface. A video camera on the vehicleobtains successive captured images of a region directly ahead of thevehicle, which contains these white lines. A plurality of areas withineach captured image are selectively examined to measure their respectivebrightness levels, with these areas being predetermined as correspondingto areas of the road surface that are normally outside the white lineswhen the vehicle is travelling along the center of a traffic lane. Therespective brightness values of these areas are measured, and theexposure of the vehicle-mounted camera is controlled based on theresults.

On the case of capturing images for use in recognition of a targetobject (i.e., a 3-dimensional object) such as a preceding vehicle, itwould be possible to perform exposure control of the camera based onmeasuring the brightness of the road surface as described above, sincenormally that brightness is relatively stable. However the brightnessvalues of various vehicles may differ substantially, so that in theprior art, such exposure control has been performed based upon measuringthe brightness of the preceding vehicle which is to be detected. Howeverwhile such exposure control is being performed, sudden changes in thelevel of measured brightness may occur, since the preceding vehicle canarbitrarily enter or leave the field of view of the camera, or anothervehicle (e.g., having a different level of brightness) may suddenlyenter the field of view of the camera, by cutting-in ahead of the localvehicle.

In the prior art, it has not been possible to control the cameranexposure to respond sufficiently quickly to such abrupt changes in thelevel of brightness that is being measured, so that stable and accuratecontrol of the cameran exposure has been difficult to achieve.

SUMMARY OF THE INVENTION

It is an objective of the present invention to overcome the aboveproblem by providing an exposure control apparatus for a vehicle-mountedcamera, which is capable of stable measurement of the brightness of atarget object and hence provides improved control of the cameranexposure, for a purpose of obtaining images to be used in processing forrecognition of the target object.

The invention provides an exposure control apparatus for controlling theexposure of an electronic digital camera installed on a vehicle (e.g.,by controlling the shutter speed, etc., of the camera), with the camerabeing disposed to capture an external scene as an image formed of anarray of picture elements having respective luminance values. Theapparatus is configured (e.g., by being provided with exposurerelationship data stored beforehand in a non-volatile memory) to convertthe luminance values of an image to corresponding brightness values ofthe external scene that is being captured by the camera, with theconversion being executed based on the current exposure condition(shutter speed, etc.,) of the camera.

The apparatus is characterized in comprising extraction circuitry forextracting (from each image captured by the camera) a set of pictureelements which constitute a target object-use region of the image, foruse in measuring the brightness of a target object (in general, apreceding vehicle) located ahead of the local vehicle, and a second setof picture elements which constitute a road surface-use region of theimage, for use in measuring the brightness value of the road surface.The camera exposure is controlled based upon brightness values obtainedfor the road surface-use region and for the target object-use region, incombination.

By comparison with prior art methods of exposure control of a such avehicle-mounted camera, whereby only the brightness of the target objectis measured and used in controlling the cameran exposure, the presentinvention enables more stable exposure control to be achieved.

Furthermore in certain circumstances, such as when the external scenevaries between sunlight and shade conditions, or the local vehicleenters or exits from a tunnel, sudden large changes in the brightness ofthe road surface can also occur. However with the present invention, dueto the fact that exposure control is based upon measuring both thebrightness of a target object (if present) and also the brightness ofthe road surface, greater stability of exposure control can be achievedunder various conditions.

Furthermore with the present invention, such improved stability isachieved without requiring an excessive size of a brightness measurementregion within each image (i.e., a region whose picture elements are usedfor deriving a measured brightness value to be applied in controllingthe cameran exposure). As a result, the improved stability of exposurecontrol can be achieved without causing a significant increase in theprocessing load by comparison with other types of exposure controlapparatus.

The target object-use region and the road surface-use region arepreferably located at respective fixedly predetermined positions withineach image. This reduces the possibility of erroneous recognition ofobjects and greater stability of exposure control, and so providesimproved performance by comparison with a prior art type of exposurecontrol apparatus in which the target object-use region is varied inaccordance with the recognition processing that is being performed.

The target object-use region is preferably formed with a shape whichsuccessively increases in horizontal width, along a direction towards anupper part of the road surface-use region.

In addition, the target object-use region is preferably formed such thatthe uppermost part of that region is located at the FOE (focus ofexpansion) position in the image. This serves to prevent detection ofthe brightness of the sky, buildings, etc., so that more stable exposurecontrol can be achieved, i.e., fluctuations in the detected brightnessdue to such objects can be prevented.

An exposure control apparatus according to the present invention can beadvantageously implemented by processing performed in accordance with aprogram executed by a computer.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a general block diagram of a vehicle system incorporating anembodiment of an exposure control apparatus;

FIG. 2 shows examples of exposure control maps for use with theembodiment;

FIG. 3 is a flow diagram of exposure control processing executed by theembodiment;

FIG. 4 shows diagrams for use in describing a way of setting luminancecontrol target values which are used with the embodiment;

FIG. 5 is a diagram for describing a dead zone of brightness values;

FIG. 6 is a flow diagram of processing for deriving a capture-objectivebrightness value, indicative of brightness in an external region aheadof a vehicle in which the embodiment is installed;

FIG. 7 illustrates the form of a brightness measurement region of animage;

FIG. 8 illustrates thinning-out of picture elements from respectivelines of the brightness measurement region;

FIG. 9 illustrates exclusion of highest-brightness and lowest-brightnesspicture elements from each of respective lines of the brightnessmeasurement region;

FIG. 10 is a diagram showing an example of distribution of brightnessvalues in an image captured by a vehicle-mounted camera;

FIG. 11 illustrates the derivation of average brightness values ofrespective lines of picture elements in the brightness measurementregion;

FIG. 12 illustrates a manner in which the strength of time-axisfiltering applied to successively obtained average brightness values ofrespective lines of picture elements in the brightness measurementregion is determined;

FIG. 13 shows diagrams illustrating an operation of judging whether ornot time-axis filter is applied to respective lines of picture elementsin the brightness measurement region;

FIG. 14 shows diagrams illustrating an operation of judging whether ornot a capture-objective brightness value for use in exposure control isobtained by low-pass filter processing of successively obtained values;

FIG. 15 is a graph which is used in evaluating the magnitude offluctuations in successively obtained capture-objective brightnessvalues in relations to a half-width value of the dead zone; and

FIG. 16 shows an example of luminance control maps for enabling scenebrightness measurement to be performed over a wide range of externalscene brightness values.

DESCRIPTION OF PREFERRED EMBODIMENTS

An embodiment of an exposure control apparatus for a vehicle-mountedcamera will be described in the following referring to the drawings.

1. Overall Configuration

FIG. 1 is a block diagram showing the general configuration of anexposure control apparatus 10 and its relationship to other componentsof a vehicle system. The exposure control apparatus 10 is installed in avehicle (the “local vehicle”) and is connected to a vehicle-mounteddigital video camera (referred to in the following simply as the camera)21 and to a unified judgement section 22, with the unified judgementsection 22 being connected to a warning indication generating section 23and a steering control ECU 24 as shown. Data obtained by the exposurecontrol apparatus 10 based on the contents of captured images of a sceneahead of the local vehicle are used in warning indication processing andsteering control processing that are executed by the unified judgementsection 22. Specifically, recognition processing is applied to the imagedata, to detect a preceding vehicle as a target object which may belocated ahead of the local vehicle, and the results of the recognitionprocessing are used for warning indication and steering controlpurposes.

The exposure control apparatus 10 consists of a CPU 11, a memory section12, an image interface 13 which transfers data of successive capturedimages from the camera 21 to the CPU 11, and a communication interface14 for executing communication with the unified judgement section 22.The memory section 12 includes a non-volatile memory such as a ROM(read-only memory, not shown in the drawings) having programs and dataincluding a plurality of luminance control maps (described hereinafter)stored therein beforehand, and a RAM (random-access memory, not shown inthe drawings) and data registers, for storing and processing data of aplurality of images which have been successively captured up to thecurrent point in time.

The camera 21 is installed within the passenger compartment of the localvehicle at a fixed position (for example, beside the rear view mirror),and captures successive images (i.e., as respective video signal frames)of a region of the road ahead of the vehicle. When installed in thevehicle, the orientation of the camera 21 is adjusted such as to set aspecific image capture range with respect to the direction ofadvancement of the vehicle.

The camera 21 incorporates a usual type of CCD or CMOS image sensor,together with a video amplifier, A/D (analog-to-digital) converter, etc.When an image is captured by the image sensor, as an analog signalexpressing successive luminance values, the video amplifier applies aspecific amount of gain to the analog signal, which is then converted tosuccessive digital values (luminance values of picture elements) by theA/D converter, and stored as data in the memory section 12. The CPU 11then reads out and processes the image data, operating separately onrespective picture lines of the image, where each picture line is ahorizontal row of picture elements (horizontal scan line) of the image.

The image interface 13 transfers the picture element values, togetherwith horizontal and vertical synchronizing signals of the image, fromthe camera 21 to the CPU 11. The CPU 11 determines respective imagepositions corresponding to each of the picture elements, based upon thehorizontal and vertical synchronizing signals. The picture elementvalues are then stored in the memory section 12 in correspondence withposition information specifying the respective locations of the pictureelements within the image.

The CPU 11 processes the image data to perform recognition of a targetobject such as a preceding vehicle which may appear in the capturedimages. Based on the recognition processing results, the CPU 11 suppliesposition information concerning any target object to the unifiedjudgement section 22 via the exposure control apparatus 10.

In addition, the CPU 11 controls the camera 21 such as to appropriatelycapture images of the scene ahead of the vehicle. Specifically, the CPU11 adjusts the frame rate and the exposure parameters of the camera 21,by generating corresponding adjustment commands and supplying these tothe camera 21 as camera control command values. In the following it isassumed that the exposure parameters of the camera 21 are the shutterspeed and video amplifier gain.

The communication interface 14 enables communication between the CPU 11and the unified judgement section 22, for transferring to the unifiedjudgement section 22 the above-described information concerning resultsof target object recognition. Based on this information, the unifiedjudgement section 22 judges whether there is a danger of collisionbetween the local vehicle and a target object. When it is judged thatsuch a danger exists, the unified judgement section 22 controls thewarning indication generating section 23 to generate a warningindication to the vehicle driver. If the danger is judged to be above apredetermined level, then in addition to generating a warningindication, the unified judgement section 22 also instructs the steeringcontrol ECU 24 to perform appropriate steering control of the vehicle.Specifically, this may be control whereby the amount of steeringassistance that is applied to the steering mechanism is adjustedappropriately, or whereby the steering mechanism is controlled to beautomatically driven such as to avoid the danger of collision.

2. Outline of Exposure Control

The cameran exposure control operation of this embodiment will besummarized in the following. The exposure control apparatus 10 data hasstored therein beforehand expressing a plurality of characteristicsreferred to in the following as luminance control maps. Each of thesecorresponds to a specific exposure condition of the camera 21 (specificcombination of shutter speed and video amplifier gain), and expressesthe relationship between the brightness of an external scene ahead ofthe vehicle which is captured as an image by the camera 21, andresultant luminance values of picture elements of the image. The pictureelement luminance values are supplied from the camera 21 as respectivedigital values.

FIG. 2 shows an example of such a plurality of luminance control maps,with values of external scene brightness plotted along the horizontalaxis and image luminance (picture element luminance values, as obtainedfrom the camera) along the vertical axis.

In the example of diagram (a) of FIG. 2, if the image luminance (e.g.,average of a plurality of picture element values) is a value V (referredto herein as a luminance control target value, which is predetermined asbeing an appropriate value of image luminance), when a capture-objectivebrightness value (measured as described hereinafter) is B and theluminance control map 11 is being used, then this is a condition inwhich the cameran exposure parameters (shutter speed, video amplifiergain), determined by the luminance control map 11, are correctly set.

Referring to diagram (b) of FIG. 2 however in which the luminancecontrol map No. 7 is being used, with the capture-objective brightnessvalue B being as shown, the image luminance value deviates from thetarget value V, i.e., takes the value K, so that the cameran exposure isnot correctly set. In that case the apparatus performs exposure controlby selecting the luminance control map No. 13, so that the imageluminance will be restored to the target value V. Exposure control isthereby applied such as to maintain the image luminance close to anappropriate value, irrespective of variations in brightness of the scenethat is captured by the camera.

The luminance control target value is determined in accordance with theluminance control map which is currently selected, i.e., there is apredetermined relationship between the luminance control map numbers andthe luminance control target values, as described hereinafter.

With this embodiment, instead of measuring the scene brightness based onall of the picture elements of a captured image from the camera 21 it isderived based on a fixedly predetermined part of each image, having aspecific shape, location and size, referred to as the brightnessmeasurement region.

The exposure control apparatus 10 of this embodiment basically performsexposure control in accordance with the following sequence of operations(1) to (4).

(1) Determination of Luminance Control Target Value

The luminance control target value is determined in accordance with thecurrently selected luminance control map, based on the aforementionedpredetermined relationship, and varies between a day value and a nightvalue. To ensure that the control conditions do not change abruptly, theluminance control target value varies only gradually during eachtransition between the day value and night value.

(2) Calculation of Dead Zone

A dead zone (illustrated in FIG. 5) of brightness values is determined,as described hereinafter.

(3) Derivation of Capture-Objective Brightness Value

Two adjoining regions within each captured image constitute theaforementioned brightness measurement region with this embodiment, i.e.,a road surface-use region for measuring the brightness of the roadsurface, and a target object-use region for measuring the brightness ofa preceding vehicle (when present), as shown in FIG. 7. With thisembodiment, respectively different forms of weighted-averagingprocessing are applied to these two regions, and a capture-objectivebrightness value is obtained based on average brightness values that arerespectively calculated for the two regions.

(4) Control of Amplifier Gain and Shutter Speed

If the capture-objective brightness value obtained by operation (3) isfound to be outside the dead zone, an appropriate other one of theluminance control maps is selected to be used, based upon the luminancecontrol target value determined in operation (1) and upon thecapture-objective brightness value obtained in operation (3), asdescribed above referring to diagram (b) of FIG. 2. The exposurecondition (shutter speed and amplifier gain) of the camera 21 is thenadjusted in accordance with the newly selected luminance control map.

3. Processing Executed by CPU

The CPU 11 periodically (e.g., once in every 100 ms) executes aprocessing routine in accordance with a stored program, as exposurecontrol processing. In this processing, a capture-objective brightnessvalue is derived based on data of one or more images that have beensuccessively acquired up to the current point in time from the camera 21and stored in the memory section 12. Based on this capture-objectivebrightness value, the luminance control map is changed if necessary, andthe cameran exposure parameters (shutter speed, amplifier gain) adjustedaccordingly. This processing will be described referring to the flowdiagram of FIG. 3.

When processing begins, the CPU 11 first (step S110) determines aluminance control target value. Specifically, a correspondencerelationship (shown as the full-line characteristic in the diagram (c)of FIG. 4) is stored beforehand, relating luminance control map numbers(e.g., the map numbers 1 to 17 shown in FIG. 2), plotted along thehorizontal axis, to luminance control target values which are along thevertical axis. Based on that correspondence relationship, a luminancecontrol target value is derived in accordance with the number of theluminance control map which is currently being used.

At the first execution of the processing routine of FIG. 3 (whenoperation of the system is started), a predetermined one of theluminance control maps is selected to be utilized, and the correspondingluminance control target value is obtained.

The correspondence relationship of FIG. 4( c) is derived by averagingthe correspondence relationships of FIGS. 4( a) and 4(b), which arerespectively shown as a dotted-line characteristic and as a broken-linecharacteristic in FIG. 4( c). FIG. 4( a) is a relationship betweenluminance control map numbers (plotted along the horizontal axis) toroad surface luminance control target values (along the vertical axis)which is appropriate for the aforementioned road surface-use region ofthe brightness measurement region (i.e., an image region containing apart of the road surface that is close to and directly ahead of thelocal vehicle). FIG. 4( b) is a corresponding relationship which isappropriate for the target object-use region of the brightnessmeasurement region (an image region which is some distance ahead of thelocal vehicle and may contain a target object such as a precedingvehicle).

Hence with this embodiment, each luminance control target value is notsimply determined as being appropriate for an image region in which atarget object is to be recognized, but instead is derived as acombination of target values that are appropriate for a target objectand for the road surface, respectively.

When the average scene brightness is low (in general, at night),luminance control maps having low numbers will be selected for use,whereas when the average scene brightness is high (during daytime), mapshaving high numbers will be utilized. With this embodiment asillustrated in FIG. 4, the relationship between the luminance controltarget values and map value numbers is predetermined such that a lowluminance control target value is selected during night operation and ahigher luminance control target value is selected during daytimeoperation. This is done to ensure that the apparatus will functioncorrectly even when large-scale increases in image luminance occurduring night-time operation (e.g., due to light received from sourcessuch as headlamps of oncoming vehicles, etc.).

Also as shown, there is a gradual transition between the night-useluminance control target value and the daytime-use luminance controltarget value, to prevent abrupt changes in image luminance. Since theluminance control target value is selected in accordance with theluminance control map which is currently in use, the gradual transitionis achieved by appropriately relating the luminance control targetvalues to the luminance control map numbers.

Next in step S120, the dead zone is calculated. This is a range ofbrightness values for use in judging whether it is necessary to adjustthe cameran exposure (select another luminance control map). The deadzone is used to prevent unnecessary frequent changes in the exposurecondition. Specifically as shown in FIG. 5, designating the luminancecontrol map that is currently being used as map N, and designating thecorresponding luminance control target value (obtained in step S110) asV, the dead zone is defined as a scene brightness range extendingbetween the intersections of the luminance control target value V withthe two adjacent luminance control maps (N−1) and (N+1) (i.e., mapswhose numbers immediately precede and immediately succeed that of thecurrently selected luminance control map).

Next in step S130, processing is performed to obtain thecapture-objective brightness value. This is based on converting thepicture element luminance values of the brightness measurement region(i.e., specific fixed region within the image) to correspondingconverted brightness values by using the luminance control map which iscurrently selected, and will be described referring to the flow diagramof FIG. 6.

Firstly in step S131, the picture element values of the brightnessmeasurement region are acquired, in units of picture lines. As shown inFIG. 7, the brightness measurement region of this embodiment is formedof a trapezoidal region referred to as the target object-use region, formeasuring the brightness of a preceding vehicle (i.e., a region locatedsome distance ahead of the local vehicle, at a position where apreceding vehicle may appear in the image) and a rectangular regionreferred to as the road surface-use region, corresponding to a part ofthe road which is located close to and immediately in front of the localvehicle, and which serves for measuring the brightness of the roadsurface. The image luminance value is measured as a combination ofvalues that are derived from the target object-use region and the roadsurface-use region.

Specifically, the road surface-use region has a vertical dimension(height dimension) corresponding to an area that extends approximately 7to 27 meters ahead from the front of the local vehicle, and a widthdimension (lateral dimension) determined such as to contain the twowhite lines which are located respectively at the right and left sidesof a traffic lane in which the local vehicle is running.

The uppermost part of the target object-use region is set at the FOE(focus of expansion) position for the camera 21. The width of thatuppermost part is made equal to the typical azimuth extent (±10°) of aregion scanned by a millimeter-wave radar apparatus which may beinstalled in the local vehicle, for scanning the scene ahead of thevehicle with radar waves and judging the position, shape, speed, etc.,of preceding objects based on resultant reflected radar waves.

The trapezoidal shape of the target object-use region successivelywidens towards the upper part of the road surface-use region, i.e., itis formed of picture lines that are of successively increasing length,whereas the road surface-use region is formed of full-width picturelines (corresponding to the full horizontal angle of view of the camera21). This shape of the target object-use region is used to ensure thatthe cameran exposure can be rapidly adjusted when another vehicle cutsin ahead of the local vehicle, i.e., to provide a seamless transitionbetween detecting the brightness of the road surface and detecting thebrightness of a preceding vehicle.

Since the external region (in the scene ahead of the local vehicle) thatis beyond the FOE will generally contain features such as sky,buildings, etc., which are not relevant as target objects, it is ensuredthat these are excluded from the captured images, and so will not havean adverse effect upon exposure control.

To reduce the data processing load, thinning-out of picture lines isperformed (i.e., with one out of each of successive pluralities ofpicture lines of the image being omitted) when extracting (from the mostrecently captured image) picture elements constituting the luminancemeasurement region. In the road surface-use region, thinning-out ofpicture lines is performed at spacings which are approximately identicalto one another with respect to distance from the local vehicle. That isto say, the higher the positions of the lines within the luminancemeasurement region, the smaller is made the proportion of lines omittedby the thinning-out processing. In the target object-use region, thethinning-out is performed at regular spacings, i.e., the spacing betweenlines that are omitted by the thinning-out processing is held constant.

In addition, periodic thinning-out of picture elements within each lineof the brightness measurement region is also performed, as indicatedconceptually by the dotted-line portions in FIG. 8. With thisembodiment, this periodic omission of respective picture elements (i.e.,of luminance values corresponding to these picture elements) isperformed at identical spacings within each picture line.

The luminance values of the picture elements of the brightnessmeasurement region are the converted to respectively correspondingbrightness values (i.e., indicative of brightness values in the externalscene) by using the currently selected luminance control map andluminance control target value. Referring for example to diagram (b) ofFIG. 2, assuming that a picture element value (luminance value) obtainedfrom the camera 21 is K, then as indicated by the directions of thearrows, the corresponding converted brightness value is obtained as B byapplying the currently selected luminance control map No. 7.

Next in step S132, for each picture line of the brightness measurementregion, the picture elements are sorted in order of brightness value,then a fixed number of maximum-brightness picture elements and a fixednumber of minimum-brightness picture elements of that line are excludedfrom further processing.

Assuming each of these fixed numbers is greater than one, the term“fixed number of maximum-brightness picture elements” as used in thisdescription and in the appended claims signifies “the maximum-brightnesspicture element and one or more picture elements having successivelylower brightness than the maximum-brightness value”. Similarly, the term“fixed number of minimum-brightness picture elements” signifies thelowest-brightness picture element and one or more picture elementshaving converted brightness values that are successively higher than theminimum value.

Although with this embodiment, the above exclusion processing isperformed based upon judging-converted brightness values of pictureelements, it would also be possible to perform the exclusion processingbased upon judging the luminance values, i.e., the picture elementvalues as obtained from the camera 21.

In the case of a road surface having a light coloration, such as aconcrete surface, dark regions on the surface (such as portions repairedwith coal tar, or joints in the roadway) are an obstruction to reliablymeasuring the brightness of the road surface. In the case of a dark roadsurface, e.g., formed of asphalt, white lines that are formed on thesurface will similarly hinder reliable measurement of the brightness ofthe road surface. This is illustrated by the example of the distributionof brightness values of picture elements, for the case of a forward-viewimage of a road, shown in FIG. 10. With this embodiment, since highestand lowest brightness values of the brightness measurement region areexcluded from further processing as described above, such problems dueto excessively light or excessively dark regions on the road surface canbe overcome.

In the case of a part of the road surface that is close to (directlyahead of) the local vehicle, it is possible to comparatively reliablydistinguish excessively high or low brightness values resulting fromwhite lines, coal tar patches, etc., on the road surface. However in thecase of a part of the road surface that is distant from the localvehicle, it becomes difficult to distinguish such regions. For thatreason, the farther the distance represented by the image position of apicture line (i.e., the higher the location of that line within thebrightness measurement region) the smaller is made the number of pictureelement values that are excluded from the line by the exclusionprocessing described above. In the case of the picture linescorresponding to the most distant part of the brightness measurementregion, no picture element values are excluded.

Next in step S133 as illustrated in FIG. 11, for each of the remainingpicture lines of the brightness measurement region, the average of theconverted brightness values of the picture elements of the line iscalculated. The resultant respective average values are designated asB_(i,t), where “i” denotes the position of the corresponding line withinthe brightness measurement region in a range from 1 to L, counting fromthe top of the brightness measurement region (as illustrated in FIG.11), i.e., 1≦i≦L. The subscript portion “t” denotes the time-axisposition of a picture element (spatial-domain) average value, e.g.,expressed as a sequence number within a series of images successivelycaptured at periodic time points up to the current time point.

By excluding the highest and lowest luminance values from this averagingprocessing it is ensured that, for each of the picture lines of thebrightness measurement region, the (spatial) average brightness valuesof respective lines will vary in a more stable manner over time.

Next in step S134, for each of the L picture lines of the brightnessmeasurement region, buffering is performed of the respective averageluminance values that have been obtained for that picture line in aplurality of successively obtained images, using a buffer interval of(t˜t−T). That is, for each of the picture lines, a set of(spatial-domain) average values which have been previously successivelycalculated and stored at respective time points are acquired (read outfrom memory) and set in buffer registers, to be subjected to averagingcalculation. These buffered average values can be expressed as:

1st line: B_(1,t) . . . B_(1,t−T)i-th line: B_(i,t) . . . B_(i,t−T)L-th line: B_(L,t) . . . B_(L,t−T)

If for example the buffering interval is 4, then for each of the picturelines in the range 1 to L, the corresponding respective averagebrightness values that have been stored in the memory section 12 forfour successive images are acquired as the buffered average values forthat picture line.

Next in step S135, time-axis filtering (i.e., smoothing by averagingprocessing) is applied to each of selected picture lines of thebrightness measurement region (these picture lines being selected asdescribed hereinafter). The time-axis filtering is performed byobtaining, for each of the selected picture lines, the average of thebuffered values that have been acquired in step S134, i.e., assuming acontinuous set of L lines;

1st line: B_(1,t) . . . B_(1,t−T)→F_(1,t)i-th line: B_(i,t) . . . B_(i,t−T)→F_(i,t)L-th line: B_(L,t) . . . B_(L,t−T)→F_(L,t)

It can be expected that there will be only a small degree of variationin the average brightness values of picture lines corresponding to aregion that is close to (i.e., is immediately ahead of) the localvehicle, since the brightness of such a region will generally bedetermined by reflection of light from the road surface. Hence, littleor no time-axis filtering is applied to picture lines of such a part ofthe brightness measurement region. However in the case of picture linescorresponding to a region that is distant from the local vehicle (i.e.,is close to the FOE), there may be large amounts of time-axis variationsin the successive average brightness values that are obtained for thesepicture lines. These variations can result from effects such as pitchingof the local vehicle while light received from headlamps of opposingvehicles is affecting the brightness measurement region, thereby causinglarge changes in the successive average brightness values that aremeasured for these picture lines corresponding to a distant region.

For that reason, when time-axis filtering as described above is appliedto a picture line corresponding to a region that is close to the FOE, acomparatively long buffer interval is used, for example corresponding toapproximately 700 ms, i.e., averaging is performed using a large numberof successively obtained values (large value of T).

This is made possible since with this embodiment, time axis filteringcan be applied individually to respective picture lines of thebrightness measurement region.

The above selective application of time-axis filtering to picture lineaverage brightness values in accordance with distance from the localvehicle is illustrated in FIG. 12. As indicated, the greater thedistance of an imaged region (that is, the higher the position of thecorresponding picture lines within the captured image), the higher ismade the effectiveness of the time-axis filtering against noise(dispersed fluctuations in brightness), that is to say, the greater ismade the degree of smoothing that is applied against time-axisvariations. Conversely, no time-axis filtering is applied to the averagebrightness values of picture lines in the part of the brightnessmeasurement region that is closest to the local vehicle.

However it is also necessary that the apparatus be able to rapidlyfollow sudden changes in the scene brightness, in particular, the roadsurface brightness, which can occur when the local vehicle enters orleaves a tunnel, etc. Hence for each of the picture lines of thebrightness measurement region, the time-axis filtering is selectivelyapplied in accordance with the form of variations in the successiveaverage brightness values obtained for that line. This is done in orderto suppress fluctuations in the successive capture-objective brightnessvalues while at the same time achieving a fast reaction to suddenchanges in external scene brightness. This processing is applied to eachof the picture lines of the brightness measurement region.

Specifically with this embodiment, if the successive average brightnessvalues that are obtained for a picture line are found to be changinggradually over time as in the example of diagram (a) of FIG. 13, i.e.,along a trend, without dispersion of values, then time-axis filtering isnot applied. That is to say, the average value obtained for that pictureline in the most recently captured image is used directly in calculatingthe image luminance value (that calculation described hereinafter).

If it is found that dispersed fluctuations are occurring in the averagevalues obtained for a picture line, as illustrated in diagram (b) ofFIG. 13, then time-axis filtering by weighted median filtering isapplied to the successive average values. In all other cases, such aswhen the average values obtained for that picture line are successivelyvarying as illustrated in diagram (c), time-axis filtering bynon-weighted averaging is applied.

The term “weighted median filtering” as used herein signifies anaveraging calculation in which greater weight is given to newer datathan to older data.

The above processing performed in step S135 will be described morespecifically in the following.

Assuming for example that T is 4, where the buffer interval is(t−0˜t−T), the average brightness values of the i-th picture line withina buffer interval will be assumed to have the following magnituderelationships:

B _(i,t−1) <B _(i,t−3) <B _(i,t−2) <B _(i,t−4) <B _(i,t−0)

If either of the relationships of expression (1) below is satisfied, itis judged that dispersed fluctuations exceeding a predeterminedamplitude are occurring in the successive average values obtained forthe picture line, i.e., if the absolute difference between the newestvalue and the mid-point value exceeds the half-width (DZW/2) of the deadzone multiplied by the time separation (T/2) between these values. Inthat case, weighted median filtering is applied.

$\begin{matrix}{\frac{B_{i,{t - 0}} - B_{i,{t - 2}}}{{DZW}/2} > {\frac{T}{2}\mspace{14mu} {or}\mspace{14mu} \frac{B_{i,{t - 0}} - B_{i,{t - 2}}}{{DZW}/2}} < {- \frac{T}{2}}} & (1)\end{matrix}$

If either of the relationships of expression (2) below is satisfied,then it is judged that a gradual variation (a trend) is occurring in thesuccessive average brightness values of that picture line, so thattime-axis filtering is not applied, i.e., if the absolute differencebetween the newest value and the mid-point value exceeds the width (DZW)of the dead zone multiplied by the time separation (T/2) between thesevalues. Similarly, time-axis filtering is not applied if either of therelationships of expression (3) below is satisfied, i.e., if theabsolute difference between the newest value and the oldest valueexceeds the width (DZW) of the dead zone multiplied by the timeseparation (T) between these values.

$\begin{matrix}{\frac{B_{i,{t - 0}} - B_{i,{t - 2}}}{DZW} > {\frac{T}{2}\mspace{14mu} {or}\mspace{14mu} \frac{B_{i,{t - 0}} - B_{i,{t - 2}}}{DZW}} < {- \frac{T}{2}}} & (2) \\{\frac{B_{i,{t - 4}} - B_{i,{t - 0}}}{DZW} > {T\mspace{14mu} {or}\mspace{14mu} \frac{B_{i,{t - 4}} - B_{i,{t - 0}}}{DZW}} < {- T}} & (3)\end{matrix}$

In all other cases, time-axis filtering by non-weighted averaging isapplied.

Next in step S136, as shown by equation (4) below, weighted-averagingprocessing is applied to the set of average brightness values (ofrespective picture lines) obtained by the selectively applied time-axisfiltering of step S135. The result of this weighted-averaging processingwill be referred to as the preliminary capture-objective brightnessvalue.

In equation (4), F_(i,t) denotes the average brightness value of apicture line, and W_(i) denotes a weighting value which is set for thepicture line, for example as follows.

The preliminary capture-objective brightness value is obtained as acombination (with this embodiment, an average) of average valuesobtained for the picture lines of the target object-use region and forthe picture lines of the road surface-use region. The brightness valueswithin the road surface-use region (close to the local vehicle) arerelatively stable, while those of the target object-use region are morevariable. For that reason, when applying equation (4) to the pictureline average brightness values of the road surface-use region, therespective weighting values W that are assigned in equation (4) aresuccessively decreased in accordance with increasing closeness of thepicture line (i.e., of the region represented by the picture line) tothe local vehicle. Conversely, when applying equation (4) to the pictureline average values of the target object-use region, the value of W isdecreased in accordance with decreasing distance of the picture line(i.e., of the region represented by the picture line).

$\begin{matrix}{B_{{IMG\_ Temp},t} = {\sum\limits_{i = 1}^{L}{W_{i} \times F_{i,t}}}} & (4)\end{matrix}$

Next (step S137), a plurality of capture-objective brightness valuesthat have been successively obtained up to the current point areevaluated, to determine the extent of variation of these values. If theamplitude of the variations is within a predetermined limit, then thepreliminary capture-objective brightness value is subsequently used inperforming exposure control. If the extent of variation exceeds thelimit, then low-pass filtering processing (described hereinafter) isapplied and the result of this filtering is used in performing exposurecontrol.

This low-pass filtering processing is performed to prevent brightnesshunting.

Operation then proceeds to step S140 of FIG. 3.

Applying low-pass filtering to obtain the capture-objective brightnessvalues can cause a lowering of response speed, so that this filtering isapplied only when it is judged that these values are fluctuatingexcessively. The allowable limit of amplitude of variations of thesuccessive capture-objective brightness values is determined based onthe width of the dead zone, as described in the following.

Processing relating to the above low-pass filtering is performed in thefollowing sequence of operations, in which P designates the number ofprecedingly obtained capture-objective brightness values that are usedin evaluating the extent of variation of the capture-objectivebrightness values:

[1] Buffering (storing in data registers) of capture-objectivebrightness values that have been successively measured at periodic timepoints up to the current point (buffer interval: t˜t−P):

B_(IMG#)_Temp,t . . . B_(IMG#)_Temp,t−P

[2] Respective differences between each of these capture-objectivebrightness values and the immediately-precedingly derivedcapture-objective brightness value are calculated, as shown by equation(5) below (buffer interval: 0˜P−1):

Diff₀ =B _(IMG) _(—) _(Temp,t) −B _(IMG) _(—) _(Temp,t−1) . . .Diff_(P−1)=B_(IMG) _(—) _(Temp,t−(P−1)) −B _(IMG) _(—) _(Temp,t−P)  (5)

[3] The number of alternations in that series of capture-objectivebrightness values is then calculated, i.e., the number of changes insign between adjacent difference values (that is, between each pairDiff_(i) and Diff_(i−1) within the set of difference values Diff₀ . . .Diff_(P−1))

[4] The average absolute magnitude of the variations is evaluated inrelation to the half-width DZW/2 of the dead zone. Specifically, ifexpression (6) below is satisfied, then it is judged that C=1.0 (where Cis a parameter in equation (8) below). If expression (7) below issatisfied, then the value of C is obtained from the graph of FIG. 15.

$\begin{matrix}{\frac{\sum\limits_{j = 0}^{P - 1}{A\; B\; {S\left( {Diff}_{j} \right)}}}{P} < \frac{DZW}{2}} & (6) \\{\frac{\sum\limits_{j = 0}^{P - 1}{{ABS}\left( {Diff}_{j} \right)}}{P} \geq \frac{DZW}{2}} & (7)\end{matrix}$

[5] Low-pass filtering is then selectively applied, in accordance withequation (8) below, to obtain a capture-objective brightness value(B_(IMG,t)) for use in exposure control. That is to say, if the value ofC is obtained as 1, then the preliminary capture-objective brightnessvalue which was obtained in step S136 is subsequently used directly inexposure control. Otherwise (C<1), a low-pass filtering calculation isperformed using at least one precedingly obtained capture-objectivebrightness value, and the result of this LPF processing is used inexposure control. With this embodiment, the low-pass filteringcalculation consists of multiplying the preliminary capture-objectivebrightness value by C and the immediately precedingly obtainedcapture-objective brightness value by (1−C), and summing the results,i.e.:

B _(IMG,t) =C×B _(IMG) _(—) _(Temp,t)+(1−C)×B _(IMG,t−1)  (8)

This completes the processing of step S130 of FIG. 3. Next, in step S140of FIG. 3, a decision is made as to whether the capture-objectivebrightness value obtained in step S130 is within the dead zone. If it isjudged that the capture-objective brightness value is within the deadzone, the processing is ended. If the capture-objective brightness valueis judged to be outside the dead zone range, step S150 is then executedin which a luminance control map is selected (as described hereinabovereferring to diagram (b) of FIG. 2) in accordance with thecapture-objective brightness value obtained in step S130 and theluminance control target value which was determined in step S110.Exposure control is then performed by setting the camera video amplifiergain and shutter speed in accordance with the selected luminance controlmap. Execution of the processing is then ended.

The embodiment has been described above assuming that each of theluminance control maps have linear characteristics. However the camera21 may be operated in a HDR (high dynamic range) mode, in which therange between minimum and maximum luminance values of the pictureelements corresponds to a wider range of scene brightness values than ina normal mode. In that case, the luminance control maps may become ofthe form shown in FIG. 16, with bends formed in parts of the mapcharacteristics. As a result of these non-linearities of the luminancecontrol map characteristics, complex limitations may arise in the valuesof shutter speed and amplifier gain that can be utilized. For thatreason it is desirable that the degree of change in the extent ofbending, between adjacent luminance control maps, is made small.

By using such luminance control maps for HDR operation, when a suddenlarge change in external scene brightness occurs (for example when thelocal vehicle enters a tunnel) the time which elapses until appropriatecontrol of the cameran exposure is achieved can be reduced.

Effects Obtained

As can be understood from the above, with the exposure control apparatus10 of this embodiment, picture elements constituting a brightnessmeasurement region are extracted from each of successive captured images(step S131), with the brightness measurement region being formed of atarget object-use region and a road surface-use region. The brightnessof a target object (e.g., preceding vehicle, which is located ahead ofthe local vehicle and is required to be detected through recognitionprocessing applied to the captured images) is measured based on valuesobtained for the target object-use region, while the brightness of theroad surface is measured based on values obtained for the roadsurface-use region. As a result, greater stability of exposure control(that is, control of the exposure of a camera which is required toobtain images expressing a target object) can be achieved than ispossible with prior art types of apparatus which perform exposurecontrol based only on detecting the brightness of a target object.

Furthermore, greater stability of exposure control is achieved undervarious conditions such as when the local vehicle enters or leaves atunnel, or passes through areas of bright sunlight and shade, or when apreceding vehicle suddenly enters or exits from the field of view of thecamera, etc. However these advantages of the invention are obtainedwithout requiring a significant increase in processing load.

In addition with the present invention, stability of exposure control isenhanced by situating the uppermost part of the target object-use regionat the FOE position of each image, thereby ensuring that extraneouslight (e.g., from the sky) will not affect brightness measurement.

In the appended claims, extraction circuitry recited therein correspondsto the CPU 11 in executing the processing of step S131 of FIG. 6, whileexposure control circuitry recited therein corresponds to the CPU 11 inexecuting the processing of step S150 of FIG. 3.

It should be noted that the invention is not limited to the embodimentdescribed above, and that various modifications or alternativeembodiments could be envisaged, which fall within the scope claimed forthe invention. For example, the configuration of the brightnessmeasurement region is not limited to that of the described embodiment,and can be formed as appropriate for a particular application.

1. An exposure control apparatus for exposure control of an electronicdigital camera installed on a vehicle, said camera disposed to capturean external scene as an image comprising an array of picture elementshaving respective luminance values, said image expressing a targetobject located upon a surface of a road extending ahead of said vehicle,said exposure control apparatus comprising circuitry configured toconvert said luminance values to corresponding brightness values of saidexternal scene, based upon a predetermined relationship between anexposure condition of said camera, said luminance values, and saidbrightness values; wherein said exposure control apparatus comprises:extraction circuitry configured to extract from said image a firstplurality of picture elements to constitute a target object-use regionfor use in measuring a brightness value of said target object and asecond plurality of picture elements to constitute a road surface-useregion for use in measuring a brightness value of said road surface, andexposure control circuitry configured to control said cameran exposurebased upon said brightness values of said road surface-use region and ofsaid target object-use region, in combination.
 2. An exposure controlapparatus according to claim 1, wherein said road surface-use region andsaid target object-use region are located at respective predeterminedpositions within said image.
 3. An exposure control apparatus accordingto claim 2, wherein said target object-use region is formed with a shapewhich successively increases in horizontal width, along a directiontowards an upper part of said road surface-use region.
 4. An exposurecontrol apparatus according to claim 1, wherein an uppermost part ofsaid target object-use region is located at a FOE (focus of expansion)position within said image.
 5. An exposure control program to beexecuted by a computer, for implementing respective functions of anexposure control apparatus for exposure control of an electronic digitalcamera installed on a vehicle, said camera disposed to capture an imageof a scene ahead of said vehicle as respective luminance values of anarray of picture elements, and said exposure control apparatuscomprising means for converting said picture element luminance values tocorresponding brightness values of said external scene; wherein saidexposure control apparatus comprises: extraction means for extractingfrom said image a first plurality of picture elements to constitute atarget object-use region for use in measuring a brightness value of saidtarget object, and a second plurality of picture elements to constitutea road surface-use region for use in measuring a brightness value ofsaid road surface, and exposure control means for controlling saidexposure based upon said brightness values of said road surface-useregion and of said target object-use region, in combination.